A revised evaluation framework for region of interest detectors using artificial 3d-scenes

In this project, we introduce an evaluation method for regions of interest detectors in images of general 3d-scenes addressing some issues of the framework by Mikolajczyk et al. (2005) and the extension to non-planar scenes by Fraundorfer and Bischof (2005). By generating more reliable ground truth and test images from detailed POV-Ray scenes, we were able to take the influence of discontinuities and occlusion into account. We re-evaluate the repeatability under viewpoint change of the Harrisand Hessian-Affine region detectors by Mikolajczyk and Schmid (2004), an edge based (EBR) and an intensity based (IBR) region detector by Tuytelaars and Van Gool (2004), the maximally stable extremal region (MSER) detector Matas et al. (2002) and the Difference of Gaussian (DoG) detector by Lowe (1999). In contrast to Fraundorfer and Bischof (2005), we regard the scale invariant DoG detections to be circular regions instead of keypoints. We discuss the influence of discontinuities in 3d-scenes in general, and evaluate each detector’s tendency to extract regions on such discontinuities.